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TITLE OF THE INVENTION 



Quantization and Compression of Information in a Direct Acyclic Graph 



BACKGROUND OF THE INVENTION 



17 L Fieldof the Invention 



18 



19 The invention relates to compressing information stored in a scene graph 

20 used in a multi-media presentation. 
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1 2. Related Art 
2 

3 MPEG-4 (Moving Pictures Expert Group) is a digital bit stream format 

4 used to broadcast or multicast ("netcast") multimedia presentations. Elements such as 

5 HTML objects. Flash animations, audio/visual streams, Java scripts and similar objects 

6 can be included in an MPEG-4 scene. The MPEG-4 scene is constructed as a direct 

7 acyclic graph of nodes (also referred to as a scene graph) arranged in a hierarchical tree. 

8 Grouping nodes are used to construct the scene structure. Children of grouping nodes are 

9 used to represent the various multimedia objects in the scene. These children may also 

10 have siblings that such that each sibling is associated with a multimedia object. Each 

11 node includes a list of fields that define the particular behavior of the node. Taken 

12 together, the nodes in a scene graph represent the layout, presentation, interactions and 

1 3 animation of a multimedia scene. 
14 

15 One problem associated with using scene graphs to store information is 

16 that a scene graph for a relatively complex scene is very large. A correspondingly large 

17 memory is required to store all the scene graphs in a given presentation. The available 

18 bandwidth may not be sufficiently large, thus necessitating uncomfortably long download 

19 times. 
20 

21 One approach to solving this problem involves inserting a quantization 

22 parameter node at the top of the scene graph (that is, in a position that is relatively 

23 proximate to the root node) or at one or more other locations in the graph. A quantization 
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1 parameter node is a specialized node that shows how scene parameters (such as color, 

2 angle, scale, rotation and others) are used to specify an aspect of the scene graph or a 

3 portion of the scene graph are to be quantized. However, the degree of compression that 

4 results from this approach in not optimal because the locations where the quantization 

5 parameter nodes are placed are not responsive to the values being quantized. For 

6 example, in very small scenes, the cost (as measured by the number of bits) of specifying 

7 the quantization parameter may exceed the number of bits saved in the scene by including 

8 the quantization parameter node. In very large scenes, the degree of compression may be 

9 less than optimal because an insufficient number of quantization parameter nodes have 

10 been inserted, such that the quantization parameters include a maximum and minimum 

1 1 quantization range for extremely disparate values. In short, the problem of determining 

12 an efficient placement of quantization nodes, particularly with respect to an acceptable 

1 3 error range, remains unsolved. 
14 

15 SUMMARY OF THE INVENTION 
16 

17 In a first aspect of the invention, the size of a scene graph and the 

18 corresponding amount of memory required to store the scene graph can be reduced by 

19 selective placement of quantization parameter nodes in a scene graph. Unlike the prior 

20 art of placing the quantization parameter node at the top of the scene graph or at the top 

21 of each subtree, a technique is presented for traversing a scene graph so as to determine 

22 the most efficient placement of quantization parameter nodes. 
23 
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1 The scene graph is traversed depth first to establish an order and then 

2 traversed in reverse order. At each node, a calculation relating to (1) the relative cost of 

3 inserting a quantization parameter node, and (2) the relative savings that result from 

4 insertion of a quantization node is performed. Quantization parameter nodes are 

5 selectively placed in response to a result of these calculations. Locations in the scene 

6 graph where subtrees of the graph include clustered values (that is, values that fall within 

7 a particular range) are identified and quantization parameters that are responsive to these 

8 clustered values are inserted. This selective placement of quantization parameters 

9 reduces the size of a scene graph substantially. The scene and all of it's various audio- 

10 visual components can be transmitted and presented using less memory because a small 

11 range of values is used for each portion of the graph controlled by a particular 

1 2 quantization parameter node. 
13 

14 In a second aspect of the invention, the maximum degree of acceptable 

15 error value (designated herein as Smax) is chosen for each quantization type. This error 

16 value limits the number of quantization parameter nodes that can be placed in a scene 

17 graph. While the insertion of quantization parameter nodes increases the error, this 

18 increase remains less than a preselected maximum acceptable value. 
19 

20 BRIEF DESCRIPTION OF THE DRAWINGS 
21 

22 Figure 1 is a block diagram showing a system for quantization and 

23 compression of information in a scene graph. 
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1 Figure 2 is a block diagram showing a scene graph and quantization nodes 

2 that are selectively placed in optimal positions. 
3 

4 Figure 3 is a flow diagram showing a method for using a system for 

5 quantization and compression of information in a scene graph. 
6 

7 Figure 4 is a flow diagram of a method for traversing a scene graph used in 

8 a system for quantization and compression of information. 

9 

10 Figure 5 is a block diagram showing an exemplary scene graph with two 

1 1 subtrees and potential quantization parameter nodes. 
12 

13 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

14 

15 In the description herein, a preferred embodiment of the invention is 

16 described, including preferred process steps, materials and structures. Those skilled in 

17 the art would realize, after perusal of this application, that embodiments of the invention 

18 might be implemented using a variety of other techniques not specifically described, 

19 without undue experimentation or further invention, and that such other techniques would 

20 be within the scope and spirit of the invention. 
21 

22 
23 
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1 Lexicography 

2 

3 The following terms relate or refer to aspect of the invention or it's 

4 embodiments. The general meaning of each of these terms is intended to be illustrative 

5 and in no way limiting. 

6 

7 o Quantization - as used herein, the term * 'quantization" describes a process in which a 

8 range of values in a scene graph is represented using a fixed number of bits. The 

9 quantization process takes place prior to representing the scene graph in a compressed 

10 format. When the compressed scene graph is to be uncompressed, a reverse 

1 1 quantization process is used to return the parameter values to an approximation of 

1 2 their original values. 
13 

14 o Quantization parameter node - as used herein, the phrase "quantization parameter 

15 node" refers to a special node in a scene graph that specifies how sections of the scene 

16 graph or subtrees in the scene graph are quantized. 
17 

18 o Scene graph - as used herein, the term "scene graph" is a tree structure that includes 

19 a set of nodes that are placeholders for information regarding the display of an object 

20 in an MPEG-4 scene or other mixed media presentation. An exemplary scene graph 

21 may include a root node associated with an object and a set of dependent nodes 

22 associated with various properties of that object such as color, transparency and 
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1 texture. Additional nodes and subtrees can also be included. Taken by itself, the 

2 scene graph is static and acts as a container for information. 

3 o Scene — as used herein, the term "scene" refers to a set of objects and other elements 

4 (for example, sprites) that are present at any one point in time during a multi-media 

5 display. 
6 

7 System Elements 
8 

9 Figure 1 is a block diagram showing a system for quantization and 

10 compression of information in a scene graph. 
11 

12 A system 100 includes an originating server 110, a terminal 120 and a 

13 communication link 130. 

14 

15 The originating server 110 includes a binary scene encoder 112, a 

16 rasterizer 114, and a processor, a memory, and sufficient server software to transmit a 

17 media stream (such as an MPEG-4 presentation) to a terminal 120. 
18 

19 The binary scene encoder 112 includes a computer program 113 for 

20 generating a scene graph, traversing that graph and determining a heuristically optimal 

21 placement of quantization parameter nodes. The term "heuristically optimal" means 

22 optimal to a degree possible using a particular technique for examining a set of 

23 possibilities that cannot be examined exhaustively. 
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1 The rasterizer 114 includes a processor and a computer program for 

2 generating a bit map and a set of pixels that are responsive to information generated by 

3 the computer program 113. In a preferred embodiment, the drawing process implemented 

4 by the rasterizer 1 14 is optimized for rendering MPEG-4 data. Upon generating a set of 

5 pixels, the rasterizer 1 14 sends the set of pixels to the terminal 120. 
6 

7 Although described herein as a single device, the rasterizer 120 and to the 

8 binary scene encoder 1 10 may be incorporated into multiple devices or may be situated at 

9 different originating servers 1 10 that are coupled by a communication link 130. 
10 

11 The terminal 120 is under the control of a user 122. The terminal 120 

12 preferably includes a buffer for storing media and sufficient circuitry or software for 

13 presenting the media stream to a user 122. The terminal 120 receives the media stream, 

14 buffers and decodes the stream, and presents it to the user 122. In one embodiment, the 

15 terminal 120 may receive different media streams from rasterizer 114. The different 

16 media streams are integrated at the terminal 120 so as to comprise a single presentation 

17 fora viewer 122. 
18 



19 Various embodiments of the terminal 120 include a computer and monitor, 

20 or a television and set-top box, among others. 
21 

22 The communication link 130 can include a computer network, such as an 

23 Internet, intranet, extranet or a virtual private network. In other embodiments, the 
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1 communication link 130 can include a direct communication line, a switched network 

2 such as a telephone network, a wireless network, a form of packet transmission or some 

3 combination thereof. All variations of communication links noted herein are also known 

4 in the art of computer communication. In a preferred embodiment, the originating server 

5 110 and the terminal 120 are coupled by the communication link 130. 
6 

7 Figure 2 is a block diagram showing quantization parameter nodes that are 

8 selectively placed in optimal positions in a scene graph. 
9 

10 System 200 includes a first scene graph 210 and a second scene graph 220. 

11 The first scene graph 210 is converted into the second scene graph 210 using the 

12 computer program 113 for determining the optimal placement of quantization parameter 

13 nodes. 
14 

15 The first scene graph 210 is responsive to an MPEG-4 scene. It includes a 

16 set of nodes, shown here as nodes A - K. The scene graph 210 and set of nodes is 

17 exemplary. Other scene graphs may a different number of nodes or a different structure. 
18 

19 A node in a scene graph is a parent node if it has other nodes that descend 

20 from it. Thus, the parent nodes in scene graph 210 are nodes A, D, and E. 
21 
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1 A node is a child node if it descends from a parent node. Thus, nodes B, 

2 C, and D are children of A. Nodes F, G, and H are children of D, Nodes I, J, and K are 

3 children of E. 

4 

5 A node may be designated as both a parent and a child if it descends from 

6 a node and has nodes that depend from it. Thus, nodes D and E are both parents and 

7 children. 
8 

9 Nodes are considered to be siblings if they descend from the same parent. 

10 Thus, nodes B, C, D, and E are siblings because they all descend from A. Similarly, 

1 1 nodes F, G, and H are siblings, as are nodes I, J, and K. 
12 

13 Each node in scene graph 210 is associated with a scene construct such as 

14 a rectangle. Each node also includes various other parameters such as may relate to color, 

15 position and other parameters shown in table 1. 
16 

17 
18 
19 
20 
21 
22 
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None 


This parameter is used when there is no 
quantization. 


PositionSD 


This parameter is used for 3D positions of 
objects. 


Position 2D 


This parameter is used for 2D positions of 
objects. 


TextureCoordinate 


This parameter is used to show texture 
coordinates. 


Angle 


This parameter is used for angles. 


Scale 


This parameter is used for scales in 
transformations. 


Interpolator Keys 


This parameter is used for interpolator keys 
and MFFloat values. 


Normals 


This parameter is used for normal vectors. 


Rotations 


This parameter is sued to shown rotations 

of objects. 


ObjectSizeSD 


This parameter includes values for 3D 
object sizes. 


ObjectSize2D 


This parameter includes values for 2D 
object sizes. 


Linear Quantization 


This parameter includes values for the 
maximum, minimum and number of bits. 


Coord Quantization 


This parameter includes lists of coordinates 
of points, colors and texture. 


Color 


This parameter is used to show color and 
intensity. 



1 Table 1 shows the types of values that can be included in the nodes of a 



2 scene graph. 
3 

4 Scene graph 220 includes a set of nodes that correspond to the nodes in the 

5 first scene graph 210. As with the first scene graph 210, these nodes are designated as 

6 nodes A - K. Unlike scene graph 210, scene graph 220 also includes two quantization 

7 parameter nodes. These quantization parameter nodes are designated as X and Y. 
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1 Similar to the first scene graph 210, the nodes in the second scene graph 

2 220 are associated with a type such as a rectangle. Each node also may include values 

3 relating to the parameters included in Table 1. 
4 

5 A quantization parameter node affects the siblings to the right of the 

6 quantization node and the children of those siblings. A node may be affected by only one 

7 quantization parameter node. Thus, in the second scene graph 220, the quantization 

8 parameter node X affects nodes D, F, G, and H. Quantization parameter node Y affects 

9 nodes E, I, J, and K. 
10 

1 1 Computer program 1 13 transforms scene graph 210 into scene graph 220 

12 by insertion of the quantization parameter nodes. The location for insertion of the 

13 quantization parameter nodes is chosen so as to result in the optimal savings of bits that 

14 can be achieved with respect to a predetermined maximum error. The predetermined 

15 error is chosen so the perceptible degradation in the visible presentation of media is 

16 negligible. Using a predetermined error allows an author to specify how much error is 

17 tolerable in the presentation. It is important to note that allowing quantization error does 

18 not imply that there will be visual degradation since the error may be imperceptible. 
19 

20 In one embodiment, the predetermined error may be derived from the 

21 screen resolution of a presentation element used in the presentation of the multi-media 

22 presentation. In other embodiments, the predetermined error may be a fixed percentage 
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1 of the range of values for one or more quantization types, or a percentage of differences 

2 of successive related values. 
3 

4 A quantization parameter node includes maximum and minimum values 

5 for each quantization type included in the nodes that it affects. For example, if the nodes 

6 affected by quantization parameter Y include color, rotation, angle and scale, then the 

7 quantization parameter Y will include maximum and minimum values for color, 

8 rotations, angle and scale, along with a number of bits that are used to quantize the range 

9 of values between the maximum and minimum. 
10 

1 1 Method of Use 
12 

13 Figure 3 is a flow diagram showing a method for using a system for 

14 quantization and compression of information in a scene graph. 
15 

16 A method 300 includes a set of flow points and a set of steps. In one 

17 embodiment, the system 100 performs the method 300, although the method 300 can be 

18 performed by other systems. Although the method 300 is described serially, the steps of 

19 the method 300 can be performed by separate elements in conjunction or in parallel, 

20 whether asynchronously, in a pipelined manner, or otherwise. There is no particular 

21 requirement that the method 300 be performed in the same order in which this description 

22 lists the steps, except where so indicated. 
23 
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1 At a flow point 310, the system 100 is ready to begin performing a method 

2 300. 

3 

4 At a flow point 315, an error value is selected for each quantization type 

5 that is used in the scene graph that is included in Table 1 . Error values for quantization 

6 types that are not found in the scene graph are not selected. 

7 

8 As noted supra^ the error value may be derived from a number of sources, 

9 including the screen resolution of a presentation element, a fixed percentage of the range 

10 of values for one or more quantization types, a percentage of differences of successive 

1 1 related values or some other standard. 
12 

13 At a step 320, the scene graph is traversed depth first to determine an order 



14 of the nodes. After determining this order, the scene graph is traversed in the reverse of 

15 this order. The following step 325 is performed at each node during this reverse traversal. 

16 A detailed example of ordering and reverse traversal is shown in figure 4. 
17 

18 At a step 325, a pair of calculations for every pair of successive siblings in 

19 the scene graph is made. For these calculations, let C(T,QP) equal the cost of inserting a 

20 quantization parameter node QP before node T. Let tl and t2 represent the first and 

21 second siblings, respectively, in a pair of successive siblings. Let QPl and QP2 be 

22 quantization parameters nodes placed before tl and t2, respectively. Let QP3 be a 
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1 quantization parameter node that includes the maximum and minimum of values in both 

2 tl and t2. Given the foregoing, the calculations are: 
3 

4 A = C(t2, QP3) 

5 B = C(tl,QPl) + C(t2,QP2) 
6 

7 In a step 330, a sub-tree of a scene graph is quantized. The nodes of the 

8 sub-tree are examined and the maximum and minimum values for the quantization types 

9 are computed over the whole sub-trees. An error bound value for each of the quantization 

10 types is used to compute the number of bits of quantization which will result in a 

1 1 quantization error less than the bound. 
12 

13 The quantization process that takes place in this step involves inserting a 



14 quantization parameter node to the left of node tl if the value of A is greater than the 

15 value of B as determined in step 325. This quantization node will affect all of the 

16 siblings to the right of QPl and all of the siblings' children. After inserting QPl, the 

17 calculations are repeated for the next node in the tree. The order of the nodes for which 

1 8 this calculation is performed is described more fully in figure 4. 
19 

20 If the value of A is less than the value of B, then steps 325 and 330 are 

21 repeated for the next pair of nodes in the reverse depth-first order. These steps are 

22 repeated and the process of comparing the relative cost of inserting quantization 
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1 parameter nodes and inserting quantization parameter nodes continues until the entire 

2 scene graph is traversed. 
3 

4 The steps of the method 300 may be repeated for the remaining scenes in a 

5 multi-media presentation until quantization parameter nodes are inserted throughout the 

6 presentation where ever it is optimal to insert them. 
7 

8 Figure 4 is a flow diagram of a method for traversing a scene graph used in 

9 a system for quantization and compression of information. 
10 

11 A method 400 includes a set of flow points and a set of steps. In one 

12 embodiment, the system 100 performs the method 400, although the method 400 can be 

13 performed by other systems. Although the method 400 is described serially, the steps of 

14 the method 400 can be performed by separate elements in conjunction or in parallel, 

15 whether asynchronously, in a pipelined manner, or otherwise. There is no particular 

16 requirement that the method 400 be performed in the same order in which this description 

17 lists the steps, except where so indicated. 
18 

19 In a step 405, the system 100 is ready to begin performing a method 400. 

20 In one embodiment, the traversal begins with the root node of a scene graph. In other 

21 embodiments, the traversal can begin with anywhere in the scene graph. 
22 
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1 In a step 410, the scene graph is traversed to establish an order. The 

2 ordering begins with the left most node and proceeds through the siblings of that node 

3 moving from left to the right. If a sibling has children, the children of the sibling are 

4 traversed before moving on to the sibling. This process is continued until the nodes in the 

5 tree have been ordered. 
6 

7 Referring back to figure 2, the traversal order of the nodes in scene graph 

8 210 would be A, B, C, D, F, G, H, E, I, J, and K. 
9 

10 In a step 415, the scene graph is traversed in the reverse of the order that 

1 1 was determined in the previous step. In this reverse traversal, the last node in the order 

12 that was established in step 410 will be first and the first node in that order will be last. 
13 

14 Referring back to figure 2, the order of the reversal traversal of scene 

15 graph 210 would be K, J, I, E, H, G, F, D, C, B, A. 
16 

17 During this reversal traversal, a determination is made whether to insert a 

18 quantization parameter node to the left of the node being traversed. This determination is 

19 made for every node except for the right most node in a group of siblings. 
20 

21 The reversal traversal is continued until the scene graph is completely 

22 traversed. The process begins again at step 410, when a new scene graph in the multi- 

23 media presentation is traversed to establish an order. 
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1 Figure 5 is a block diagram showing an exemplary scene graph with two 

2 subtrees and potential quantization parameter nodes. 
3 

4 A system 500 shows a first scene graph 510 and a second scene graph 520. 



5 These two scene graphs are exemplary scene graphs for the purpose of illustrating method 

6 300 and method 400. 
7 

8 Similar to the scene graph in Figure 2, scene graph 510 and scene graph 

9 520 are responsive to an MPEG 4 scene. As described in figure 2, they include parent 

10 nodes, child nodes, sibling nodes and quantization parameter nodes. Subtrees are 

1 1 designated tl and t2. 
12 

13 The first scene graph 510 includes a quantization parameter node 

14 designated as QP3. QP3 affects all the nodes in tl and t2. The second scene graph 520 

15 includes two quantization parameter nodes, designated as QPl and QP2. QPl affects all 

16 of the nodes in tl. QP2 affects all of the nodes in t2. QPl, QP2 and QP3 show the 

17 potential location of quantization parameter nodes. 
18 



19 During the performance of methods 300 and method 400, the cost of scene 

20 graph 510 is compared to the cost of scene graph 520. As noted in step 335 of method 

21 300, the following calculations are made: 
22 

23 A= C(t2, QP3) 
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1 B=C(tl,QPl) + C(t2,QP2) 

2 

3 If A > B, QPl is inserted at the tree at tl. If A < B, then the traversal 

4 continues as shown in Figure 4 and the process is continued. This pair of calculations is 

5 made for every pair of successive siblings in a scene graph (using the ordering described 

6 in figure 4) so as to detennine a heuristically optimal placement of quantization parameter 

7 nodes based upon the relative cost of inserting a node at a particular location. 
8 

9 Alternative Embodiments 

10 

1 1 Although preferred embodiments are disclosed herein, many variations are 

12 possible which remain within the concept, scope and spirit of the invention; these 

13 variations would be clear to those skilled in the art after perusal of this application. 
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